KOSTANT HOMOLOGY FORMULAS FOR OSCILLATOR MODULES 

OF LIE SUPERALGEBRAS 
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Abstract. We provide a systematic approach to obtain formulas for characters and 
Kostant u-homology groups of the oscillator modules of the finite dimensional general 
linear and ortho-symplectic superalgebras, via Howe dualities for infinite dimensional 
Lie algebras. Specializing these Lie superalgebras to Lie algebras, we recover, in 
a new way, formulas for Kostant homology groups of unitarizable highest weight 
representations of Hermitian symmetric pairs. In addition, two new reductive dual 
pairs related to the above-mentioned u-homology computation are worked out. 
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1. Introduction 

1.1. Inspired by the Borel-Weil-Bott theorem, in a classical work |Ko| Kostant com- 
puted the u-(co)homology groups for finite dimensional simple modules of a semisimple 
Lie algebra, recovering the celebrated Weyl character formula in a purely algebraic way. 
Since then Kostant's calculation has been generalized in nontrivial ways to various se- 
tups, e.g. to integrable modules of Kac-Moody algebras [GLj . to unitarizable highest 
weight modules of Hermitian symmetric pairs [En| . to finite dimensional modules of 
general linear superalgebras [Sej (and in a different way to polynomial representations 
[CZ1| ). and more recently to modules of infinite dimensional Lie superalgebras |CK| . 

On the other hand, Howe's theory of reductive dual pairs |Hll IH2| has played im- 
portant roles in the representation theory of real and p-adic Lie groups, and there have 
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been generalizations in different directions. Dual pairs (g, G) have been formulated 
systematically by the third author [Wa] between infinite rank affine Kac-Moody al- 
gebras [DJKM, K] and finite dimensional reductive Lie groups G. More recently, 
dual pairs between finite dimensional Lie superalgebras and Lie groups G have been 
studied in depth with application to irreducible characters of in a number of papers 
[CWH ICZ21 ICLZj . A priori, no direct link between different dual pairs (jj, G) and 
(jj, G) was expected to exist, except that the 0-modules {L(g,A(A))} and g-modules 
{L(g, A/(A))} appearing in these Howe dualities can both be parameterized by (part of) 
the same index set that parameterizes the finite dimensional simple G-modules {V^}. 
Below is a list of the reductive dual pairs used in this paper which share the same G. 



Dual Pairs I (0, G) 


Dual Pairs II (0, G) 


Dual Pairs III (g*,G) 


(fll(p + m q + n),GL(d)) 


(Ooo,GL(d)) 


(Ooo,GL(d)) 


(spo(2m|2n + l),Pin(d)) 


(boo,Pin(d)) 


(b°oo,Pm(rf)) 


(osp(2m 2n),Sp(d)) 


(c oc ,Sp(d)) 


(?oo,Sp(d)) 


(spo(2m 2n),0(d)) 


(foe, 0(d)) 


(Coo,0(d)) 



Table 1: Reductive dual pairs 



1.2. The main goal of this paper is to develop a conceptual approach to computing 
Kostant u-homology groups with coefficients in the so-called oscillator 0-modules, i.e. 
those appearing in the Howe duality (0, G) (see Column I in Table 1). Remarkably, our 
results show that the u-homology groups of these oscillator 0-modules are dictated by 
those of the corresponding integrable 0-modules, where is the infinite dimensional clas- 
sical "counterpart" (see Column II) of 0. The Howe dualities (spo(2m|2re + l),Pin(d)) 
and (b^, Pin(d)) appear to be new and are worked out in Appendix lAl 

The results in this paper provided the first supporting evidence for a super dual- 
ity between categories of 0-modules and certain categories f of ^-modules [CW3j 
(generalizing the super duality |CWZ] for type A), where Qf denotes certain finite 
dimensional reductive Lie algebras corresponding to 0. As classical Kazhdan-Lusztig 
polynomials allow an interpretation in terms of u-homology |Voj . the main results here 
may be reformulated as an equality of certain Kazhdan-Lusztig polynomials for cate- 
gories and 0/. 

In spite of the infinite dimensionality, the 0-modules above are integrable and hence 
there is a standard approach to compute their Kostant homology groups (cf. [GL^ lJu| 
ILiuj ). On the other hand, the structures of the infinite dimensional oscillator modules of 
the finite dimensional Lie superalgebras are not so well understood. In the approach 
of this paper, we use essentially only the Howe dualities (g, G) and (g, G) for the same 
G (see Columns I and II in the above Table) together with some simple combinatorial 
manipulations with the characters of the integrable g-modules to derive first a character 
formula of the oscillator g-modules in a suitable form. Then from a comparison of the 
Casimir eigenvalues of the corresponding g-module and g-module, we obtain formulas 
of the corresponding u-homology groups with coefficients in L(g, At(X)), which are 
expressed in terms of the infinite Weyl group of g, from the corresponding formulas for 
H*(u,L(g,A(A))). 
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1.3. In the paper |CK| . the Howe duality (g, G) was used together with another Howe 
duality (g s ,G) (due to (CW2| for types A, B and [LZ] for types C,D) to derive a 
u-homology formula for modules over infinite dimensional Lie superalgebras q s from 
a corresponding u-homology formula for modules of g. However, the connection be- 
tween integrable modules over infinite rank affine Kac-Moody algebras and oscillator 
representations of finite dimensional Lie (super)algebras were not suspected back then. 

On the other hand, character formulas for g-modules above were obtained in different 
ways and in different forms in earlier works jCZ21 ICLZj . In |CZ2j a key role was played 
by a difficult theorem of Enright on u-homology for unitarizable highest weight modules 
of real reductive Lie algebras established by intricate arguments involving equivalence 
of categories and nontrivial combinatorics on Weyl groups |Enj . Our present approach 
bypasses Enright's theorem. Indeed, in the special case when the Lie superalgebras 
specialize to Lie algebras (i.e. q = n = in Column I and Rows 1, 3, and 4, of Table 
1), we recover Kostant homology formulas for unitarizable highest weight modules 
of three Hermitian symmetric pairs of classical types (which can be shown by some 
combinatorial argument to be equivalent to Enright's formula). 

1.4. The paper is organized as follows. In Section [21 we review and set up notations 
for various Howe dualities involving the infinite dimensional Lie algebras g and finite 
dimensional Lie superalgebras g. In Section [3l we compute the character formulas of 
the oscillator g-modules from Howe dualities. In Section [U the Casimir eigenvalues of 
modules of g and g are computed and compared, and they are used in Section [5] to 
obtain formulas for Kostant homology groups for the oscillator g-modules. 

Generalizing the type A case in |CK| . we compute in Section [6] the character formu- 
las and Kostant homology formulas for non-integrable g*-modules at negative integral 
levels appearing in the Howe dualities (g*,G) listed in Column III of Table 1. The 
dualities (g*,G) with G = GL(d), Sp(d) and 0(d) were treated in [KR] and [Wa]. The 
new case involving Pin(cZ) is worked out in Appendix [Aj where one can also find an 
additional Howe duality (spo(2m|2n + l),Pin(d)). 

1.5. Notations. Denote by y + the set of partitions. For A € CP + we denote by ^(A) 
the length of A, and by |A| the size of A. Given d G N, a non- increasing sequence 
A = (Ai, A2, . . . , Xd) of d integers will be called a generalized partition of depth d. For a 
generalized partition A of depth d, we define 

A+ := ((Ai), . . . , (X d )), X- := ((-Ax), . . . , {-X d )), 

where here and further we set (a) = max{a, 0} for a 6 Z. Then A + is a partition and 
A~ is a non-decreasing sequence of non-negative integers. 

For a sequence of non-negative integers \x = (^1,^2, . . .) we let // = (/^, fj,' 2 , • • •) be 
the partition with //■ := > j}\. Let Z, N, and Z + stand for the set of all, 

positive, and non-negative integers, respectively. 

All vector spaces, algebras, etc. are over the complex field C. 

1.6. Acknowledgment. The authors thank Academia Sinica in Taipei, University of 
Virginia, and MSRI for hospitality and support, where part of this work was carried 
out. 
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2. Howe dualities for Lie algebras g and Lie superalgebras g 

In this section we review various Howe dualities involving infinite dimensional Lie 
algebras g and finite dimensional Lie superalgebras g. 

2.1. Infinite rank afRne Lie algebras. The infinite dimensional Lie algebra gt^ and 
its subalgebras boo, Coo, foo of types B,C,D [D JKMf IK] are well known. Our notations 
regarding these Lie algebras in this paper will be the same as those in [ CK , Section 2.2], 
and we refer to loc. cit. for explicit case-by-case description of the following standard 
terminologies based on matrix elements for gL^, E n for other types, and weights 
etc. 

• g: gtgo = cioo, boo, Coo or d^, with triangular decomposition g = g + © f) g_, 

• f): a Cartan subalgebra of g, 

• /: an index set for simple roots for g, 

• [: the Levi subalgebra of g with simple roots indexed by S = i\{0}, 

• u±: the nilradicals with g = u + © [ © u_, u_ := u, 

■ n = { ctfj | z G / }: a set of simple roots for g, 

■ n v = { a^ 7 | i G / }: a set of simple coroots for g, 

• A + (resp. A - ): a set of positive (resp. negative) roots for g, 

• A^ (resp. A^): a set of positive (resp. negative) roots for t, 

• A ± (S'): the subset of roots for u±, 

■ Ajj: the fundamental weights for foo with i G I and y G {a, b, c, t)}, 

• p c : "half sum" of the positive roots in A + , 

■ W (resp. Wo): Weyl group of g (resp. [), 

■ W£ = WS(p): the set of the minimal length representatives of the right coset 
space Wo\W of length k for with y G {a, b, c, 0}, 

• L(q, A): the irreducible highest weight g-module of highest weight A G f)*. 

2.2. Reductive dual pairs on Fock spaces. 

2.2.1. Fermionic Fock spaces. We fix a positive integer t > 1 and consider i pairs of 
free fermions ^{z) with i = 1, . . . ,£. That is, we have 

with non-trivial anti-commutation relations [Vv'\ ip7' J ] = <5ijA-+s,o- Let ^ denote the 
corresponding Fock space generated by the vacuum vector |0), which is annihilated by 
ipr'*, ips '* for r, s > 0. 

We introduce a neutral fermionic field = X)re-+z ^r- 2 ^ ^ with non-trivial anti- 
commutation relations [</> r , </> s ] = <5 r + Sj o- Denote by 5 5 the Fock space of 4>{z) generated 
by a vacuum vector that is annihilated by (f> r for r > 0. We denote by the tensor 
product of 3* and #2 . 
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2.2.2. The (Ql OQ ,GL(d)) -duality. We denote by (1 <i,j< d) the elementary d x d 
matrix with 1 in the ith row and jth column and elsewhere. Then H = E; Cen is a 
Cartan subalgebra of gl(d), while Yli<j^- e ij ^ s a Borel subalgebra of gl(d) containing 
H. An irreducible rational representation of gl(d) (or GL(d)) is determined by its 
highest weight A S ff* with (A, en) = Aj G Z (1 < i < d) and Ai > ... > A^. 
Identifying A with (Ai,...,Ad) as usual, we denote by Vq L ^ the irreducible GL(<i)- 
module of highest weight A. These irreducible modules are parameterized by the set of 
generalized partitions of depth d: 

3>(GL(d)) :={A = (Ai, . . . , A.,) | Ai eZ, A x > ... > A d }. 

Proposition 2.1. [FY] (cf. )jW&\ Theorem 3.1]) There exists a commuting action of gi^ 
and GL(d) on $ d . Furthermore, under this joint action, we have 

(2-1) 3 d = L(gUA a (A))®F G A L(d) , 

AeT(GL(d)) 

where A»(A) := dA§ + E;>i(A + )^ - E^oC^+i^ = £jLi A v 

We assume below that x n (n £ Z) and (i € N) are formal indeterminates. Com- 

n€2 



puting the trace of the operator Y\nez x n nn Yli=i z i" on both sides of (|2.ip . we obtain 



the following identity: 

d 

(2.2) [] H (1 + x n2i )(l + xr-nC 1 ) = E chL (0 [ oo, A a (A)) chV£ L(d) . 

i=l neN A60>(GL(d)) 

2.2.3. The (Cqo, Sp(d)) -duality. Let d be even and Sp(d) denote the symplectic group, 
which may be viewed as the subgroup of GL(d) preserving the non-degenerate skew- 
symmetric bilinear form on C d given by 




Here is the following k x k matrix: 

/0 ••• l\ 
••• 10 

(2.3) J fe 



1 ••• 
\1 ••• 0/ 

Let sp(d) be the Lie algebra of Sp(d). We take as a Borel subalgebra of sp(d) the 
subalgebra of upper triangular matrices, and as a Cartan subalgebra H the subalgebra 
spanned by = en — ed+i-i,d+i~i (1 < i < f). A finite dimensional irreducible 
representation of sp(d) is determined by its highest weight A E i/* with (A,ej) = A.; € 
Z+ (1 < i < i ) and Ai > ... > Ad. Furthermore each such representation lifts to an 

1 2 

irreducible representation of Sp(<i), which is denoted by ^gp^y Put 

T(Sp(d)) := { A = (Ai, . . . , Ad) | \ € Z + , Ai > . . . > Ad }, 

2 2 
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which is the set of partitions of length no more than i . 

Proposition 2.2. [Wat Theorem 3.4] There exists a commuting action of andSp(d) 

d 

on $2 . Furthermore, under this joint action, we have 

(2-4) $1= L(coo,A c (A))0y s A p(d) , 

\£T{Sp(d)) 

d 

d, 



where A<(A) := fAg + E fc >i K^k = ELi ^ 

~ <L 

Computing the trace of flneM x n n Wi=i Z V on both sides of (|2.4p . we have 



(2-5) Hll(l + x n z i )(l + x n zr 1 )= £ chL( Coo ,A c (A))chy s A p(d) . 

i=ln€N Ae?(Sp(d)) 

2.2.4. The (Doo, 0(d)) -duality. Write d G N as d = 2£ or d = 2£ + 1 with I e N. Let 
0(d) denote the orthogonal group which is the subgroup of GL(d) preserving the non- 
degenerate symmetric bilinear form on C d determined by Jd of (|2.3j) . Let so(d) be 
the Lie algebra of 0(d). We take as a Cartan subalgebra H of 00(d) the subalgebra 
spanned by ei := en — ed+\-%,d+i-i (1 < 2 < fy, while we take as the Borel subalgebra 
the subalgebra of upper triangular matrices. 

For A G H* let Aj = (A, ej), for 1 < i < ^. Then a finite dimensional irreducible 
module of so(2£) is determined by its highest weight A satisfying the condition Ai > 
• • • > A^_i > |A^| with either Aj £ Z or else Aj G ^ + Z, for all 1 < i < ^. Furthermore it 
lifts to a module of S0(2^) if and only if Aj G Z for 1 < % < I. Also a finite dimensional 
irreducible module of 5o(2£ + 1) is determined by its highest weight A satisfying the 
conditions Ai > . . . > A^ with either \ G Z + or else Aj G \ + Z+, for all 1 < i < I. 
Furthermore it lifts to a module of S0(2£ + 1) if and only if Ai G Z + , for 1 < i < £ Put 

3>(0(d)) := { A = (Ai, . . . , A d ) I A* € Z+, Ai > . . . > Ad, Ai + A' 2 < d}. 

For A G CP(0(d)), let A be the partition obtained from A by replacing its first column 
with d — X'y . 

Suppose that d = 21 and let A = (Ai, . . . , X e , 0, . . . , 0) G 7(0(21)). For X e > 0, 
the irreducible 0(2£)-module Vqm): viewed as an so(2^)-module, is isomorphic to the 
direct sum of irreducible modules of highest weights (Ai, . . . , A^) and (Ai, . . . , — Xg). 
If \( = 0, the 0(2£)-module Vq^s, viewed as an so(2£)-module, is isomorphic to 
the irreducible module of highest weight (Ai, . . . , A^_i,0), on which the element r = 
Yli^e i+i e a + e £,e+i + e e+i,e G 0(2£) \ SO(2£) transforms trivially on highest weight 
vectors. Set Vq^-, := Vq, 2 ^ <g>det, where det is the one-dimensional non-trivial module 
of 0(2£). ~ 

Suppose that d = 2£+l and let A = (Ai,. . . , X e , 0, . . . , 0) G 0>(O(2^+l)). Let V A (2m) 
be the irreducible 0(21 + l)-module isomorphic to the irreducible module of highest 
weight (Ai, . . . , A^) as an SO(2£ + l)-module, on which — 1^ acts trivially. Here is the 

d x d identity matrix. Also, we let Vqi^i+i) := ^0(2^+1) ® ^et ( c ^ e, §' P^i IH1| ). 
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Proposition 2.3. |Wa} Theorems 3.2 and 4.1] There exists a commuting action of Dqo 
and 0(d) on #2. Furthermore, under this joint action, we have 

(2-6) ^= L^oo.A'CA))®^, 

As3>(0(d)) 

Wiere A 9 (A) := dAg + E fe >i A' fc e fc . 

Suppose that d = 2£ Computing the trace of OneN x n" Y\i=i Z T 011 Q2.6P gives us 

i 

(2.7) HH(l + x n z t )(l + x n zr 1 )= chL(7j oo ,A (A))chV^ (2£) . 

Suppose that d = 11 + 1. Let e be the eigenvalue of —Id on 0(2£ + l)-modules 
satisfying e 2 = 1. From the computation of the trace of OneN x n" Oi=i Z T(~ Id) 011 
both sides of (12.61). we obtain 



(2.8) \{\{{l + ex n z i ){l + ex n zr 1 )(l + ex n )= chL(7j 00 , A a (A))ch^ (2m) . 

j=lnGN Ae0>(O(2^+l)) 

Note that chV^ 2 ^ is a Laurent polynomial in z\, . . . , zt and chV^ 2 ^ = ch Vq , while 
chV r Qj. 2f+1 ^ is the Laurent polynomial in zi, ■ ■ ■ , zt, e and chV^^-^ = e chVQ^ 2i+1 y 

2.2.5. T/ie (boo, Vm.(d))- duality. Let d be even. The Lie group Pin(<i) is a double cover 
of 0(d), with Spin(d) as the inverse image of SO(d) under the covering map (see 
e.g. [BTJ). An irreducible representation of Spin(ci) that does not factor through SO(d) 
is an irreducible representation of 50(d) of highest weight of the form 

(2.9) (Ai + ^,...,Ad_ 1 + -,A| + -), or (Ai + -,..., Xd_ x + -, -Ad - -), 

where Ai > . . . > Ad with Aj € Z + for 1 < i < i. We put 

2 

9(Pm(d)) := { A = (Ai, ■ ■ • , Ad) I Ai G Z +J Ai > . . . > Ad }. 

2 2 

For A G CP(Pin((i)), let us denote by Vp in ^ the irreducible representation of Pin(d) 
induced from the irreducible representation of Spin(d) whose highest weight is given 
by either of the two weights in (|2.9p . When restricted to Spin(cf), V^ in ^ decomposes 
into a direct sum of two irreducible representations of highest weights given by those 
in (|2.9|) . The modules in {^p^^lA £ 7(P'm(d))} are precisely those finite dimensional 
irreducible representations of Pin(d) that do not factor through 0(d). 

Proposition 2.4. [Wat Theorem 3.3] There exists a commuting action of and 
Pin(d) on ^2 . Furthermore, under this joint action, we have 

(2-10) LO^A^A))®^), 

AeT(Pin(d)) 

where A b (A) := dA b + Zk>i K £ k- 
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Computing the trace of the operator Y\ n ez x n n Yli=i Z T 011 both sides of (|2,10p . gives 



i i 



(2.11) nri( z " +z i ~ 2 )a + xnz i )(l + x n z- 1 )= chLCBoo.A^A))^^. 

i=l neN AeIP(Pin(d)) 

2.3. Formulas for u -homology groups of g-modules. Recall that the Weyl group 
W can be written as W = W W° with W° = \_\ k>0 W£. For [L e fj* and w £ W we set 
wo fj,-— w(n + p c ) - p c . 

Since we have (w o A f (A), aj) € Z + , for u; € IF and j G 5, it follows that we may 
find partitions = ((A^)i, (A^)2, • • •) and X w = ((Au,)i, (X w )2, ■ ■ •) such that woA^A) 
can be written as 

'dA-l + Ei>o( A i)i e i - Ej>o( A ^)i+i e -i> if ? = a > 
dA o + Ej>o( A ^)i e i' if jr = £>, 3, 



woA r (A) 



c. 



The following is obtained by applying the Kostant homology formula for integrable 
(=standard) modules over Kac-Moody algebra q (cf. e.g. [Jul Theorem 3.13]) and the 
Euler-Poincare principle to the complex for the corresponding Lie algebra homology 
(see [CKl Section 2.4, (2.18)]). 

Proposition 2.5. We have the following character formula: 

1 oo 

chL( ,A'(A)) = — ^(-l) fc chH fc (u-;L( ,Af(A))), 

where chHfc(u_; L(g, A ? (A))) is given by 

E we w-»(f) a A+( a: i' a; 2,...)aAs( a; o 1 » a; -i>---). */? = a > 



i,j e N. 



(2.12) 

and 

riiC 1 - n^jC 1 - ^^i) 3 »/?=&, 

Here, for a partition 7, s 7 denotes the corresponding Schur function. 

2.4. The general linear and ortho-symplectic superalgebras. Let p, q,m,n £ 
Z + . We briefly recall the general linear superalgebra gi(p + m\q + n) and ortho- 
symplectic superalgebras spo(2m|2n+l), osp(2m|2n), spo(2m|2n) (see e.g. |Klj ). which 
will be called of type a, b, c, and t>, respectively, for reasons of Howe dualities appear- 
ing later on. The following notations associated with these Lie superalgebras will be 
assumed throughout the paper. 

' : flt(p + m \<l + n ); spo(2m|2n + 1), osp(2m|2n), spo(2m|2n), or their central 
extensions in Section [H7TI 
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• f): a Cartan subalgebra of g, 

• /: an index set for simple roots for g, 

• U = e I}: a set of simple roots for g, 
•IT = { /3 V | i G / }: a set of simple coroots for g, 

• A + : a set of positive roots for g, 

• L(g, A): the irreducible highest weight g-module of highest weight A S f) . 

2.4.1. Denote by i£,P+ m \i+ n the complex superspace of dimension {jp+m\q+n) with ba- 
sis { v~ p , . . . , v _i, w-g, . . . , W-i,v±, . . . , v m ,wi, . . . , w n }, where degVi := and degWj := 
1. With respect to this basis the general linear superalgebra gl(p + m\q + n) may be 
regarded as the Lie superalgebra of complex matrices (flij), with i,j £ I p + m \q+n-> where 
Ip+m\q+n := {~Pi ■ ■ ■ j — L !>•••) "i}U{— q, . . . , — 1, 1, . . . , n}. For notational convenience 
we declare a linear ordering of I p + m \ q + n by setting 

-p<---< -l<-g<---<-T<l<---<m<T<---<n. 

For i £ I p+m \ q+n \ {n\, i + 1 means the minimum of those j £ I p + m \q+n with j > i. In 
a similar way one defines i — 1, for i £ I p + m \q+n \ {~ p}- 

Denote by E; L j, i,j £ I p+m \ q+n , the elementary matrix with 1 at the ith row and the 
jth column, and zero elsewhere. Then f) is spanned by E^k, k £ I p + m \ q + n - Let Ei and 
Sj, with i £ {— p, . . . , —1, 1, . . . , m} and j £ {—q, . . . , —1, 1, . . . ,n}, form the basis of h* 
dual to En and Ej j, respectively. We choose our Borel subalgebra to be the subalgebra 
spanned by with i < j. With respect to this Borel subalgebra, we have 

n V = { # = - £7« ( i e {-p + 1, . . . , -T } \ {-q}), 

$ = Eu-E i+w (i £ {l,...,W=T}\{m}), 

P-q = E-1,-1 + E-q-q, 0q = + En, = E mm + Ejj }, 

n = { Pi = £i-i - Ei (i = -p+ 1,. . . , -1), Pi=Ei- E i+1 (i = 1, . . . ,m - 1), 
% = ~ ^' (•?' = _<? + 1 '- • • % = 5 i ~ S j+i (i = !>• • • > n - !)> 

= £-1 — A) = S-l — £lj Pm = £m — $1 }, 

A + = { Ei - £j, Ei - Sj, 6i - Ej, Si - Sj (i < j) }. 

The associated Dynkin diagram is as follows: (® denotes an odd isotropic root) 

o o-^-o c^®-o o-^-o o 

P-p+1 P-l P-« |8-<f+l P-l Po Pi 0m -1 dm P T P~ 

2.4.2. The ortho-symplectic Lie superalgebra osp(2m|2n) is a subalgebra of gl(2m|2n) 
consisting of the linear transformations that preserve a non-degenerate even super- 
symmetric bilinear form (-|-), namely, osp(2m|2n) = osp(2m|2n)g © osp(2m|2n)j with 
05p(2m\2n) K equal to 

{A £ gl(2m\2n) K \(Av\w) = (-l) 1+Kdcgv (v\Aw) for homogeneous v,w £ C 2m l 2n }. 
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Below let (-|-) be the super symmetric bilinear form corresponding to the matrix (see 
O for J fc ) 



J2m 














J n 











The Cartan subalgebra f) of osp(2m|2n) is spanned by E{ := En — E2 m +i-i,2m+l-i and 
Ej := £jj — E 2n _j +1 2n -j+v * = 1) • • • j wi) and j = 1, . . . , n. We use the same notation 
for the restrictions of £j and <5j to the Cartan subalgebra of osp(2m|2n). Let 

n V = {/3 v = -^i - 3a, & y = ^ - (i e I m \ n \{m,n}), 

Pm = E m + Ej}, 

II = { A) = -£l - ^2, fii = £i- £i+l (i = i, . . • , TO - 1), 
/3m = £m - <5l, /% = <5j - (j = I,-- - ,n - 1)}, 
A + = { ±£i- ej, ±Si - 6j, -2Si, ±5i - 5j (i < j) }. 
The associated Dynkin diagram is as follows: 

00 




-o o — (g^ — c^- 

'h 03 0m-l 0m 0T 

a 
0i 

2.4.3. Taking the skew-supersymmetric bilinear form corresponding to the matrix 






Jm 





Jm 














J2n 



we can construct similarly the Lie superalgebra spo(2m|2n), which is isomorphic to 
osp(2n|2m). Similarly we define the Cartan subalgebra h, £j, and 6j. Let 

n V = { /3 V = -Ei, A V = Ei- E i+1 ( i G I Mn \{m,n}), 

fim = E m + Ej\ , 

n = { A) = -2ei, ft = £i - Ej+i (i = 1,... ,m- 1), 

/?m = £m - 5i, /3j = Sj - S j+ t (j = 1, . . . ,n - 1) }, 
A + = { - 2e*, ±£j - £j, ±£j - <5j, ±5i - 5j- (i < j) }. 
The associated Dynkin diagram is as follows: 

o^c^o o— ®^-o o 

00 01 02 m -l 0m 01 0— 
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2.4.4. The skew-supersymmetric bilinear form corresponding to the matrix 




gives rise to spo(2m|2n + 1). The Cartan subalgebra f) of spo(2m|2ra + 1) is spanned 

by Ei := En — E2m+i-i,2m+i-i and Ej := Ejj — -E , 2 n-j+2,2n-j+2> * = 1) • • • ) m i and 
j = 1, . . . , n. We define £», and <5j analogously. Let 

n v = {/3 v = ^!, $ = E ._ E . +1 ( ie i m]n \{m,fi}), 

Pm = E m + -Ej}, 

n = {/3o = -£i, A = £i - £i+i (i = 1, • • • , ,m - 1), 

Pm=£m- h, Pj = Sj ~ Sj +1 ( j = 1,. . . ,71 - 1) }, 

A + = { - £ 4 , -2e i5 ±£ { - ej, ±£i - 6j, -Sj, ±5i - Sj (i < j) }. 

The associated Dynkin diagram is as follows (# denotes an odd non-isotropic simple 
root): 

#^C^O O— (8^0 O 

PO Pi |8m-l /3m |% ftn^T 

2.5. Reductive dual pairs (g, G). Below we recall Howe dualities involving the Lie 
superalgebras gl(p + m\q + n), spo(2m\2n + 1), osp(2m|2n) and spo(2m|2n). 

Let d € N. Suppose that A is a generalized partition of depth d with A m+ i < n 
and Arf-p > —q. Then A^ +1 < n, and A^_ p < g (recall from Section [1.5p . Define 

Ay (A) G rj* to be 

-1 

A /( A ) : =-Efe + i-?)+ d ) £ i 

— 1 m n 

- E (cn , -i-'0«i+EV"e*+E<(A + );-n»>* i . 

j=-q i=l 3=1 

The following Howe duality was built on the special case when p = q = obtained 
in [CWTl Theorem 3.2] (also cf. [S2]). According to jCLZ] S(C P ^* <g> C d * C m l n ® C d ) 
carries a natural commuting action of Ql(p+m\q+n) and GL(d), which form a reductive 
dual pair in the sense of |HlJ. 

Proposition 2.6. [CLZ} Theorem 3.3] As a gt(p + m\q + n) x GL(d)-module we have 



S{C p ^ q * <g> C d * ©C m l n ® C d ) L{g[{p + m\q + n), A a f (X)) ® V< 



A 

GL(d) ' 
A 



where the summation is over all A E 5 (GL(d)), subject to the conditions \ m +i < n and 
Ad_p > -q. 
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Let d be even. On the superspace S(C m \ n ®C d ) we have natural actions of osp(2m|2ra) 
and Sp(d), which form a reductive dual pair (111, Section 3]. 

Proposition 2.7. [CZ21 Theorem 5.2] As an osp(2m|2n) x Sp(d) -module we have 
S(C m \ n ® C d ) - 0L(o S p(2m|2n), A}(A)) ® y s A p(d) , 

A 

where the summation is over all A G IP(Sp(d)) m'i/i A m+ i < n ; and 
A}(A) := £(A< + -d) e< + ]r«A;. - m> - -d)* 3 . 



i=i j=i 



Let d be even or odd. On the superspace S(C m ^ n <S> C d ) we have a natural action of 
spo(2m|2n) and O(ci), which form a reductive dual pair |HH Section 3]. 

Proposition 2.8. [CZ2l Theorem 5.1] Let d = 21 or U + 1 with i G Z+. 4s «n 
spo(2m|2ra) x 0(d) -module we have 

s{C m\n C dj ^ 0L(spo(2m|2n), A} (A)) ® V^ (d) , 

A 

where the sum is over all A £ IP(0(d)) lozi/i A m+ i < n, and 

m 1 n 1 

AJ(A) := £(A< + -d)e t + £«A$ - m) - -d)^, 

i=l 3=1 

The following new Howe duality is worked out in Appendix lA.il 

Proposition 2.9. (Theorem lA.ip Let d be even. As an spo(2m|2n + l) x Vm.(d) -module 
we have 

s(c 2m|2n+l ^f) L(spo ( 2m |2n + 1), Aj(A)) ® V& n(d) , 
A 

where the summation is over all A G CP(Pin(d)) mi/t A m +i < n ; and 
A}(A) := £(A, + -d) £i + £«A< - m) - -d)5 3 . 

i=l 3=1 

The simple g-modules constructed in this section will be referred to as oscillator 
modules of g. 

From now on, we mean by (g, G) and (g, G7) one of the dual pairs of type r £ {a, b, c, 5} 
in Sections 12.21 and 12.51 respectively. We let 7 S (G) be the subset of weights in 7(G) 
that appear in the (g, G)-duality decompositions. 

3. Character formulas of oscillator modules of Lie superalgebras 

In this section we derive character formulas for the oscillator representations of g, 
essentially only using the character formulas of the integrable modules of infinite di- 
mensional Lie algebra g. To that end, we introduce (trivial) central extensions of g. 
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3.1. Central extensions. Given Z € g\{p + m\q + n), we define 

fo(A, B) := Str([3, i,5e gl(p + m\q + n), 



where Str(cy) = J2i 



-9 c ii 



El 



It follows that /3-j is 



a 2-cocycle which defines a central extension of gi(p + m|g + n). We note that this 
cocycle is a coboundary, since we can construct this extension on the Lie superalgebra 
gl(p+m\q+n)(BCK , with a central element K, as follows. Define for A £ gl(p+m\q+n) 



A := A - Sti(ZA)K. 



Then [A, B] = [A, B] + fy(A, B)K. 
From now on we let 



f-Iv 



(3.1) 



















We denote by g\{p + m|q + n) the resulting central extension of gi(p + m|g + n) by the 
one-dimensional center CK. The simple roots of the derived algebra of qI(j> + m\q + n) 
are the same as those of the derived algebra of gl(p + m\q + n). The simple coroots are 



also the same except that /3g 



E_ 



1,-1 



+ £11 is replaced by E_\ _j + £n + K. 



Consider the (2m + In) x (2m + 2n) matrix of the form 



(3.2) 



z' 















°\ 


1 













2 










-In 


















In) 



The cocycle /3^/ gives rise to a central extension of gl(2m|2n) by a one-dimensional 
center CK. The induced central extensions of osp(2m|2ra) and spo(2m|2n) by a one- 
dimensional center CK are denoted by osp(2m|2n) and spo(2m|2n), respectively. 

Similarly we let spo(2m|2n + 1) stand for the central extension of spo(2m|2n + 1) by 
the one-dimensional center CK, induced from the central extension of g[(2m[2n + 1) 
associated with /3y/, where 2" is the following (2m + 2n + 1) x (2m + 2n + 1) matrix 



(3.3) 



z" 



{- 


Ira 











o\ 







Im 



















-In 

























\ 














In) 



3.2. g[{p + m\q + n). For later convenience of comparing with infinite dimensional Lie 
algebras, we introduce an additional element K, which commutes with gl(p + m\q + n) 
and GL(cf), and regard S{C p \ q * <g> C d * C m \ n ® C d ) as a module over (gl(p + m\q + n) 
Ci^) x GL(<f), where we declare that K acts as the scalar d. For Z as in (|3.ip set 



(3.4) 



Bij := Etf - Str^^OK. 
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Then © Cif gives rise to the central extension g[(p + m\q + n). Now gl(p + 

m|g + n) and GL(d) form a reductive dual pair on S{C p \ q * ® C d * ®C m l n ® C d ). 

Define A G rj by (A ,iT) = 1 and (A ,E kk ) = for all A; G ip+m^+n- For A € 
y(GL(d)) with A m+ i < n and Xd- P > —9, we define 

-l 

A)(A) := dXg - E <Wi - 

i=— p 

— 1 m n 

- E ( A ~)V; + E x i £ i + E« A+ )^ - m >^> 

j=-q i=l 3=1 

where Ag = A° - £7% ^ + EjL q Sj . 

Recall that a partition A € y + is called an (m\n)-hook partition, if A m +i < ri [BRJ. 
For an (m|n)-hook partition A, we let A^ := YliLi ^i £ i + Ej=i(^j ~~ m )$j e ^mln - 

Let £ := {£l,---,£n}, 2/ := {z/1, • • • ,2/m }, n" 1 := { »7Zj, • • • , r\Z\ }, and x" 1 : = 
{ xl}, . . . , xZ p } be indeterminates. The character of L(gl(m\n), X^), i.e. the trace of 

the operator YliLi vf" Wj=i ^f" > ^ s gi ven by the hook Schur polynomial \BR\ I5e] 
(3-5) hs x (y,0 := E s /^ yi >---> ym ) SA 7M'(£i'---'£™)- 

MCA 

Observe that hs\(y,£) = hs\>(£,y). 

Theorem 3.1. For A G CP(GL(d)) wrai/i A m +i < n and A^_ p > — g ; we /iat>e 
chL(g[(p + m\q + n), A£(A)) 

_ E^ol- 1 )^ Ewewgja) hs x+(^ V) hs \- fa -1 . 

Proof. Computing the trace of the operator TJ^ • s t xf"r^ 3 ' : 'yf ss ^f t ' t TJfc=i z fc fefe on both 
sides of the isomorphism in Proposition 12.61 where — p < i < —1, — q < j < — 1, 
1 < s < m, and 1 < t < n, we obtain 



(3.6) n n = E chL (0^+^i9+^)>^(A))chy G \ w , 



where the summation is over all A G CP(GL(d)) with A m+ i < n and A^_„ > —q. 

Recall the identity (12, 2\\ which results from the (gl^, GL(d))-duality. Now (|2.2p and 
Proposition 12.51 imply that 

[J \{{l + x n z k ){l + xl l _ n z- 1 )= chL(gl 00 ,A a (A))ch^ L(d) 

A;=lneN Ae?(GL(d)) 

~~ 2^ n n _ - 1 ^ x GL(d)- 

AeJ>(GL(d)) Hi,^ 1 X -i+l^V 
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We set £ t := x t (t = 1, . . . ,n), y s := x n+s (s G N), := Xj ( j = 0, . . . , -q + 1 ), 
and Xi := (i € — N), and rewrite the above as 



II 11(1+ xT^k ')(! + ^ 7 V ')(! + tf.**)(l + = E chV G+ ' ' ^ 



rX 

/ GL(d) 

k=li,j,s,t XeT(GL(d)) 



E^lo(- 1 ) A: Ewew° * A +(6, ■ ■ ■ , Cn, yu ■ ■ -)s x - ■ ■ ■ , r)_],x_{, . . .) 

Let w be the standard involution of symmetric functions that interchanges elementary 
symmetric functions and complete symmetric functions (e.g. [Mac! (2.7)]). Note that 
oj sends s M to s^i and recall f)3.5[) . Applying ui twice to the above identity, once on the 
variables _ 2 , • • • and another on the variables y\, jj2, ■ ■ ., we obtain 

A n ('+^)(i + ^V) = E , 

J- -I- J- -I- (l - V Zu )(\ - T^Z^ 1 ) ^ () 

k=li,j,s,t^ L UsZkA 1 X l Z k ) AeIP(GL(cf)) 

ET=o(.- 1 ) k EweW° hs X+ • • • 1/1, ■ ■ -)^ A - (Cl> • • • • • •) 

Finally setting x~_i = Vm+i = for i G N, we get an identity which shares the same 
left hand side as ()3.6p . The theorem now follows from comparison of the right hand 
sides and the linear independence of the characters {chV^ L ^|A G J > (GL(d))}. □ 

Remark 3.2. The character here, which is the trace of the corresponding operators 
with hats from gl(p + m\q + n), differs from the usual one from gl(p + m\q + n) by 

exactly a factor of ^~ 9 ...^~' L J • Similar remarks apply below to spo/osp-characters 

versus spo/osp-characters, where the power d here is replaced by |. 

Our formula here differs from the character formula obtained in |CLZ[ Theorem 5.3]. 



Remark 3.3. Denote by u> a a linear extension of the composition of maps in the proof 
of Theorem 13.11 that sends chL(gl 00 , A a (A)) to chL(gl(p + m\q + n), A" (A)). Since each 
map in the composite is either an involution u or an evaluation of some variables at 
zero, u a respects the multiplication of characters. 

3.3. osp(2m|2n). Suppose that d is even. Recalling that dsp(2m|2n) is the central 
extension induced from jj[(2m|2n) with respect to Z' in (|3.2p . we first introduce an 
element K that acts on 5(C m ' ra <g> C d ) as the scalar |, and that commutes with the 
actions of osp(2m|2n) and Sp(cZ). Setting 

A := A - Stiff A)K G osp(2m|2n) CK, A G osp(2m|2n), 

gives rise to an action of osp(2m|2n) on S(C m \ n <g> C d ). Clearly osp(2m|2n) and Sp(d) 
also form a reductive dual pair. Define an element A G f) by (A ,K) = 1 and 
{T ,Ei) = (T ,Ej) = 0, for all 1 < i < m and 1 < j < n. Further for A G 0>(Sp(d)) 
with A m+ i < re, we define 
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, Til n 

A}(A) := -Aq + tei + E< A ; " m >^' 
i=i i=l 

where Aq = Aq + YaLi e * ~~ Sj=i ^i" Then {ej, <5j, Aq} is the basis dual to {Ei,Ej, K}. 
Theorem 3.4. For A G CP(Sp(d)) with A m+ i < n, we have 

i<j<m rii< 

w/iere 77 := { r/i, . . . , rj n }, a; := {xi, . . . , x m }. 

E ^ ~ 

Proof. Computing the trace of the operator Y\.%j Vi %x j 3 Wk=\ z k° 011 both sides of the 
isomorphism in Proposition 12.71 where 1 < i < m and 1 < j < n, we obtain 

i 

W 110 7^1^^ = £ chL(cTp(2 m |2„),4(A))ch^ (d , 

fc=l i,j V 1 X ^fc A 1 &,Z fc j AeO>(Sp(d)) 

Replacing chL(g,A c (A)) in (|2.5p by the expression in Proposition 12.51 we obtain an 
identity of symmetric functions in variables x\,X2, ■ ■ ■■ We apply to this identity the 

involution on the ring of symmetric functions in xi,X2, Next, we replace x% by rji, 

i = 1, . . . , n, and then Xi +n by X{, i G N. Finally, we put Xj = for i > n + 1. Under 
the composition of those maps, we obtain a new identity which shares the same left 
hand side as (|3,7p . Comparing the right hand sides of this new identity and of ([37 



we obtain the result thanks to the linear independence of {chV^^|A G IP(Sp(d))}. □ 

We denote by w c the map which sends chL(coo, A C (A)) to chL(osp(2m|2n), A^(A)) 
and extend u; c by linearity (compare (12.51) and (13. 7ft ). Note that w c respects the multi- 
plication of characters. 

3.4. spo(2m|2n). Recall the central extension spo(2m|2n) induced from gl(2m\2n) de- 
termined by %' in (13. 2p . Introduce an element K that acts on 5(C m ' n ® C d ) as the 
scalar ^ and commutes with the actions of spo(2m|2ra) and 0(d). Putting A := 
^-Str^'^if G spo(2m|2n)0CK, for A G spo(2m|2n), defines an action of spo(2m|2ra) 
on S(C m \ n <g> C d ). Then spo(2m|2n) and 0(<i) form a reductive dual pair. Define an 
element Aq G ff by (A ,K) = 1 and (A° ,Ei) = (K ,Ej) = 0, for all 1 < i < m and 

1 < j < Tl. 

When d = 21, calculating the trace of the operator \\ i ■•qf % x^ :i ]~Ifc=i z k k 011 both 
sides of the isomorphism in Proposition 12.81 where 1 < i < m and 1 < j < n, gives 



( 3 - 8 ) \\\\ 1 + T^MI + Tz\ = S chL( S po(2 m |2n),A}(A))chV- (2 , ) , 

A m+ i<n 
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and when d = 21 + 1, the trace of the operator Ylij vf* 3 ^ 1 Yli=i z k°^~^d) gives 
. (1 - e^j^ 1 )(l - exiZk)(l - exi) 



k=l i,j 

chL(ipb(2m|2n), A a / (A))chy o A (2 , 4 , , . 



AeT(0(2£+l)) 
A m +i<n 



Here for A € 9(0(d)) with A m+ i < n, 

j m n 

A}(A) : = -AS + J] A ^ + E( A i " m ><^ 
i=l j=l 

with Ag = Ao + Er=i^-E-=i^- 

Now the next theorem follows by similar arguments as for Theorem 13.41 using (|3,8p , 
(HLH), dUTJ) and (USD and Proposition [23) 

Theorem 3.5. Pu£ 77 := {rji, . . . ,rj m } and x := { x\, . . . , x n }. For A € 7 (0(d)) with 
Am+i < ^ ; we have 

(1) If d = 21 + 1, then we have 



chL(spb(2m|2n),A° f (A)) 



, E^ot" 1 ) E^giyQQ) ^ fa j 

m< IIks^J 1 - wjX 1 - + r]iX s y 



(2) If d = 21, then we have 

chL(s^o(2m|2n),A a f (A))+chL(spb(2m|2n),A a f (A)) 



E^o(~ 1 ) fc E M eW°(B) (77, j + ^ fa 

ill<i<i< m IL^a^nC 1 - - X s X t )(l + ViXs 



-1 • 



Remark 3.6. Theorem [33] (2) gives the character of the sum of two irreducible modules. 
However, in the case when X[ = £, we have A = A, and hence \ w = X w . Thus in this 
case it actually gives the character of one irreducible module, i.e. we have an identity 
as in (1). Similar remark applies in the sequel as well when dealing with G = 0(21) and 
A = A. In particular it applies to the character formulas of Kostant homology groups 
in Theorems 15.71 and 16.51 an d also to f)5. 1 1) . 

Character formulas in forms different from Theorem 13.41 and Theorem 13.51 were also 
obtained in |CZ2[ Theorems 6.2, 6.3] using [En] Theorem 2.2]. Our approach uses Howe 
dualities involving infinite dimensional Lie algebras, thus bypassing jEnj . 

Introduce a map oj d such that u; a (chL(0oo, A°(A))) = chL(spb(2m\2n),Al(X)) if 
d = 2£+l, and w a [chL^oo, A a (A)) + diL^, A 5 (A))] = chL(spo(2m\2n), A} (A)) + 
ch.L(spb(2m\2n), A^(A)) if d = 21. Extended by linearity, w° sends either side of ([27?]) 
and (I2.8p to the corresponding side of (13. 8h and (13. 9p . respectively. 
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3.5. spo(2m|2n + 1). Recall that spo(2m|2n + 1) is the central extension induced from 
g[(2m|2n + 1) determined by Z" in (pP|) . Let if act on S(C 2m \ 2n+1 ® C%) as the 
scalar | and commute with the actions of spo(2m|2n + 1) and Pin(d). Putting A := 
A - Stv(Z"A)K G spo(2m|2n + 1) Of, for A G spo(2m|2n + 1), defines an action of 
spb(2m\2n+l) on 5(C 2m l 2n+1 (g)C5). Thenipb(2m|2n + 1) and Pin(d) form a reductive 
dual pair. Define an element Aq G h* by (A ,K) = 1 and (A ,Ei) = (A ,Ej) = 0, for 
all 1 < i < m and 1 < j •< n. 

Computing the trace of the operator Y\.ij vf' x ^ J Tlk=i z k° on both sides of the iso- 
morphism in Proposition 12.91 gives 

\ , -|,(l + ^ fc _1 )(l + ^fc) 



(3.io) niR**+*. 



(1 - XiZ k *)(1 - XiZ k ) 



k=l i,j ^ 



^ chL(spo(2m|2n + 1), A}(A))chU p A in(2£) , 

Ae3>(Pin(2f)) 

Am+l<Ii 



where 



AJ(A) := -Aq + £ A^ + - m>^, 

i=i j=i 

with^ = A;+Er=i^-E-=i^- 

Now the next theorem follows from similar arguments as for Theorem 13.41 using 
HOD, CHID, and Proposition [231 

Theorem 3.7. Put 77 := { 771, . . . , rj m } and a; := {xi, . . . ,x n }. For A G J > (Pin(ci)) wii/t 
A m +i < we /icwe 

chL(spo(2m|2n + l),Aj(A)) 

E£o(- 1 ) fc £weWj(6) hs x w (V,x) 

We denote by a; b the map which sends chL(boo,A b ( A)) to chL(osp(2m|2ra+ 1), Ajj-(A)) 
and extend uj b by linearity. 



4. The bilinear forms and Casimir operators for Lie (super)algebras 

4.1. The bilinear form ( | ) c and the Casimir operator SI of g. Recall the symmet- 
ric bilinear form (-|-) c defined on fj* in [CK\ Section 4.1], which induces a non-degenerate 
invariant symmetric bilinear form on q' = [q,q], also denoted by (-|-) c . Using this one 
defines a Casimir operator which commutes with the action of q on a highest weight 
g-module V of highest weight A, and which acts as the scalar (A + 2p c \\) c on V. The 
details here are completely analogous to Section 14.21 below, and we refer to |CK} Sec- 
tion 4.1] for more detail. 
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4.2. The bilinear form ( | ) s and the Casimir operator Q of g. Suppose first 
that g = Ql(p + m\q + n). Note that {e^, Sj } U { Aq} is the basis of b,* dual to the basis 
{E kk }U{K} off). Set 

— 1 m —1 n 

(4.1) p s := (-» - q)e l + £(1 - + (~j ~ l ) 5 3 + " ^ 

i=-p i=l i=-<? j=i 

We choose a symmetric bilinear form (-|-) s on b* satisfying 

K 

{\\si) s = -(A, En + —), i £ {-p, . . . , -1, 1, . . . , m}, 



(\\Sj) s = (A, E- fj -—), j€ {-q, . . . , -1, 1, . . . ,n}. 
We check easily that 

(ei\ej) s = -Sij, (ei\5j) s = 0, (Si\6j) s = <%. 

Also we have (Ag|£_j) s = (Ag|<$-j)« = — (Ag|^) s = -(Ag^)* = ±, for i, j > 0. Fur- 
thermore, recalling the simple roots (3 k for g, we have 

Next, suppose that q is ipb(2m|2n + 1), osp(2m|2n) or spb(2m|2n). Note that 
{e i: Sj}U{ Aq} is the basis of ff dual to {E i: Ej } U { K}, t G {b, c, 0}. We set 

'££iH + \)ei + E"=i(m - J + for r = b, 

(4-2) p s := ^ ZT= i(l " O^i + £?=i(m " i)<^ for r = c, 

X" i + TT j= M - j + l)Sj, for r = 0. 

We choose a symmetric bilinear form (-|-) s on b,* satisfying for A G b* 

(A|£j) s = {X,Ei), ie{l,...,m}, 

(X\8 j ) s = -(X,E- j ), je{l,...,n}. 
Similarly, we can check that (ej|ej) s = <5jj, (£j|<5j) s = 0, (Si\Sj) s = —Sij. Also we have 
(A | £i ) s = (A r |^-) s = 1, and (p s |A) s = ±(/3 fe |/3 fe ) s for k el. 



For i G I define 



5° •- 



Then we have 



-1, if i G {-p+ 1, • • . ,-1,-qr, 1, . . . ,m}, 

1, if ie{-g-l,... ,-1,0,1,... ,n-l}. 

— 1, if i = and r = b, 

1, if i = and r = c, 

2, if i = and r = c), 

1, if i G {1, . . . , m} and t G {b, c, t)}, 

— 1, if i G {1, . . . ,n — 1} and r G {b, c,5}. 

(A|ft) s = ^(A,/3 J v ), *€J, AGf. 
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By defining (ft~i\ftj) s := {s\s^)~ l (fti\ftj) s , we obtain a symmetric bilinear form on the 
Cartan subalgebra of g' = [g,g], which can be extended to a non-degenerate invariant 
super-symmetric bilinear form on g' such that 

(e»|7j)a = <%/4 

where ej and f i denote the Chevalley generators of g' with [ej, / J = 

Let gg be the root space of ft £ . Take up G g^ and vP G g_ /3 for /3 € A + 
such that (« Q |tt /3 ) s = 6 a p. For any highest weight g-module V, with weight space 
decomposition V = ® V^, we define Ti : V — > V to be the linear map that acts as the 
scalar (/x + 2p s |/x) s on V^. Let T 2 := 2 X)^ 6 s+ u°up. Define the Casimir operator to be 

(4.3) n : =ri + r 2 . 

It is straightforward to check the following. 

Proposition 4.1. The operator £1 commutes with the action of q on a highest weight 
module V with highest weight X, and acts on V as the scalar (A + 2p s \X) s . 

4.3. The sets of weights 7+ and [P+ + in h*. Recall the Levi subalgebra I of g with 
simple roots indexed by S. Let c G C. 

For g = doo, let 3>+ c consist of fi G f)* of the following form: 

(4.4) /i = cAg + - Cje-j , 

i>l j>0 

where ?? = (rji,rj2, ■ ■ •) and £ = (Co> Cii • • •) € (Note the shift of index for £.) We 
denote by !P^ C + the subset of which consists of \i above with r\ and £ being (n\m) 
and (g|p)-hook partitions, respectively. 

For g = b^, Coo, t>oo> let "Pf consist of a G h* of the following form: 

(4.5) ^ = cA + ^^ej, 

where Q«i,/X2, . . .) G 3 5 " 1 ". Denote by J > ^" c + the subset of !P^ C of the form above with ^ 
being an (njm)-hook partition. We put 

ceC ceC 

4.4. The sets of weights !P^ + in h*. Let A := A + U A be the set of roots of g, 
where A = — A + . Put 

S:=1\{0}. 

Let Ag := A* D (Ere5 Z ^) and ^(S) : = S± Let 

u± := Qp, l:= 0/3®^ 
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so that g = u+ © I © u_. If q = gl(p + m\q + n), then I = Ql(p\q) © gl(m\n) © OT, and 
[ = Ql(m\n) © C-fT, otherwise. The Lie superalgebras I and q share the same Cartan 
subalgebra f). 

Let c G C. For q = gl(p + m\q + n), let CP^" 1 " consist of /I G f) of the following form: 

/I = cAfj + ti£i H h r m e m + (t[ - m)5i H V (r' n - m)5 n 

- (6^-1 + ' ' ' + Z q $- q + (£1 - q)e-i + • • • + {t' p - q)e- P ) , 

where r = (ti,T2, . . .) and £ = (£i,£,2, • • •) are (m\n)- and (q\p)-hook partitions, respec- 
tively. 

When q is spb(2m|2n + 1), 6sp(2m|2n) or spb(2m|2n), we let 7^ consist of JZ G f) 
of the following form: 

/I = cA£ + t\E\ H h T m e m + (r{ - m)5i H h (r^ - m)S n , 

where r = (ti,T2, . . .) runs over (m|n)-hook partitions. We put 

4.5. Casimir eigenvalues of gl(p + m\q + n) versus those of gt^. Let g = Ql(p + 
m|g + n). Let fi G T+ c + be as in (JL4|). Set r = rj', u = (z/ 1; i/ 2 , • • •) := (Tm+l,Tm+2> • • •) 
and x = (X1,X2,---) := (Cq, Cq+i, ■ ■ •)■ Define a bijection 

by 

i?(/x) := cAg + nei H h r m e m + z/[5i H h 

- (C0<5-1 + ■ ■ ■ + ( q -l5- q + X'l^-l + • • • + Xp£-p) ■ 

Note that 

(4.6) ??(A a (A)) = A£(A), for A G 0>(GL(d)) with A m+ i < n and A d _ p > -q. 
For 77 and £ as above it is convenient to introduce the following symbols: 

(rj + 2pi\r))i :=^2 i r]i{r}i-2i), 
i>i 

(C + 2p 2 |C)2 :=E0(0-2j). 

i>o 

By [CKl Lemma 7.1] or [Macl (1.7)], for A = (Ai, A 2 , . . .) G we have 

(4.7) £ Aj(Aj - JK) = - £ A^ - 2(t - 1)). 

Lemma 4.2. For /i G J 5 /^, we have 

(4.8) (*%) + 2p 8 \0(ji)) a = (/x + 2p» c + C, 
w/iere 

C f = c 2 (AS|AS) s + 2c(p s |AS) s . 



22 CHENG, KWON, AND WANG 

In particular, for A E J'(GL(d}) with A m+ i < n and Xd- P > —q, (A n (A) + 2p c \ A a (A)) c = 
(/x + 2p e \n) e if and only if (A)(A) + 2p s \A a f (X)) s = (0(/i) + 2p s \ti{p)) s . 

Proof Let p £ :P+ + be as in (|4.4p . Using (|4. 7|) . we have 

(p + 2 Pc \p) c = ( V + 2 Pl \n)i + (C + 2p 2 \() 2 - c(\rj\ + id) 

= -(r + 2 P2 \t) 2 + (C + 2/> 2 |C) 2 - c(M + |C|). 
For convenience, put r\ = ^2 i>1 and £ = 2^j>o Now, we have 

0%) + 2p s \#(p)) s = (cA a + 0(r?) - 0(C) + 2 Ps |cAS + ~ 0(C)). 

= c\A a \A a ) s + 2c( Ps \A a Q ) s + - m + 2ps\m - m)s + mkww - m)* 

= C+ (0(r?) + 2p a |0fa)). + (0(C) - 2p s |0(C)) s - c(|r?| + \(\). 
First, we have 

(0(t 7 ) + 2 /9s |0(7 7 )) s 

= - [n(ri - 2 • 0) + • • • + r m (r m - 2(m - 1))] 

+ [^(K + 2(m - 1)) + • • • + <K + 2(m - n))] 
= - [ri(n - 2 ■ 0) + • • • + r m (r m - 2(m - 1))] 

n 

+ [^(^-2-l) + -.. + ^K-2n)] +2m^,[ 

fc=l 

(4.9) = - [ n (Ti - 2 ■ 0) + • • • + r m (r m - 2(m - 1))] 

- [^i(i/! - 2 • 0) + ^ 2 (f 2 - 2 • 1) + • • • )] + 2m^v k 

k>l 

= - [n(ri - 2 • 0) + • • • + r m (r m - 2(m - 1))] 

- [T m+ i(r m+ i - 2 • 0) + T m+2 (T m+ 2 — 2 • 1) H ] + 2m ^ r m+fc 

fc>i 

= - (r + 2p 2 |r) 2 . 
Similarly, we have 

(0(C)-2p s |0(C)) s 

= [Co(Co - 2 • 0) + • • • + - % - 1))] 

- [x'i(xi + 2(? - 1)) + • • • + xUx; + 2(g - p))] 
= [Co(Co - 2 • 0) + • • • + C 9 -i(C?-i - % - 1))] 

p 

- [ x ' 1 ( x ' 1 -2.1)) + ... + x' p (x p -2-p)]-2q^2x' k 

k=l 

= [Co(Co - 2 • 0) + • • • + C 9 -i(Cz-i - 2(g - 1))] 

+ [ X i(xi - 2 • 0)) + X2 (X2 - 2 • 1) + • • • ] - 2q £ X fc 
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= [Co(Co - 2 • 0) + • • • + - 2(g - 1))] 

+ [C g (C g - 2 • 0)) + C g +i(C+i - 2 • 1) + • ■ ■ ] - 2qJ2 ( q +k 

=(C + 2p 2 |C)2- 

This completes the proof of ()4.8p . The remaining part of the lemma follows from the 
definition of the bijection i?, ()4.6|) . and (|4.8p . □ 

4.6. Casimir eigenvalues of osp and spo versus those of too. Suppose that (jj, G) 
is (spb(2m\2n + l),Pin(d)), (6ip(2m|2n), Sp(d)) or (ipb(2m|2n), 0(d)). Let /i E 0>+ + 
be as in (|4.5p with r E {b, c, U} and c E C. Then := (y\, i^, ■ ■ ■) = (pi, P2> • • •)' i s 
of (m|n)-hook shape. Set r := (ri, T2, . . .) := (z-Wfi, ^m+2, • • •)• We define a bijection 
: ?f c + -> by letting 

1%) := cA{j + i^ei H h f m e m + t(o"i H h T^S n , 

where c = c(Aq, iT). Note that 

0(A'(A)) = A* (A), for A E U*(G). 
Lemma 4.3. For p E J 5 /^, we /mue 

+ 2 /3s |0(/i)) s = -(/i + 2p c \p) c + C, 

where C = c 2 (A£|A£) s + 2c{p\k\) s . In particular, (A f (A) + 2p c \A^(X)) c = (p + 2p c \p) c 
if and only if (A* (A) + 2p s \K){\)) s = (#(p) + 2p a \0(p)) a , for A E 0»(G). 

Proof. The equivalence of the identities follows easily once we establish the first identity. 
Let p = cAq + Yli>i ^i e i ^ *m <T ' ana - so A* := (Ml) M2> • • ■) i s °f (n|m)-hook shape. 

Case i. = spo(2m|2n+ 1). Let (-|-) c stand for the bilinear form on the dual Cartan 
subalgebra of boo. We compute 

(p + 2p c \p) c = (y^ pjEj + 2p c \} j p i €i) c + 2c(^2 pi€i\A^) c 

i>l i>l i>\ 

= ^2tn(jJH - 2i + 1) - c|/i°| = 0x° + 2pi|M°)i - (c - 1)|M°|- 

i>l 

On the other hand we have 

m n 

+ 2p s \${p)) s = C + c(Y^ mi + Yl T 'j 6 j\^o)s 
i=l j=l 



n m 

..... , N- Ay 



+ + ^ r-dj + 2y0 s | ^ + ^ rj(5 

i=l j=l i=l j=l 

= C + c\p°\ 

m n m n 

+ (%2(vi -2i + l)ei + ^(rj + 2m - 2i + l)<5j | ^ + ^ rfy 

i=l j=l i=l j=l 
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m n 



C + c\p,°\ + - 2i + 1) - T j( r j + 2m - 2j + 1) 

i=i j=i 



m n 



= c + (c-i)\fi°\ + Y, n{n - 2(» - 1)) + Y, T i( r J " 2 ( m + J ~ !)) 
t=i j=i 

m n 

= C + (c - 1)|m°I + Yj U ^ Ui ~ 2 ^ ~ 1 )) + v rn+j{v m+ j - 2(m + j 

i=l 3=1 

= C + (c - 1)| M °| - J^A*i(A*< - 2i)| = C + (c - l)|Ai°| - (M° + 2piK 

i>l 

Above we have used (|4.7p in the third to last identity. 
Case 2. q = asp(2m|2n). By ([477]) . we have 

+ 2p c \p) c = Y^iiVi ~ 2i) - 2c|^°| = -(i/ + 2/9 2 [^) 2 - 2c|/x°|, 
i>l 

where (-|-) c is the bilinear form on the dual Cartan subalgebra of Cqo. 
On the other hand, 

= c 2 (A< |A<) S + 2c( Ps |A<) s + + 2p a |0(/i)) a + 2c(AS|0(/z)) a 
= C + (tf(/i) + 2 /0s |tf( / u)) s + 2c| / u o |. 

And by the same argument as in ()4.9|) . we can show that 

+ 2 Pa \0{n)) a = {v + 2p 2 \u) 2 . 

Case 3. g = ipb(2m|2n). By (f¥77|) . we have 

(ju + 2p c |/i) c = ^J/U;(/Uj ~ 2{i - 1)) - c\n°\ = -(v + 2/0i [ i/) j - c\p°\, 
i>l 

where (-|-) c is the bilinear form on the dual Cartan subalgebra of Oqo. 
On the other hand, 

0%) + 2/9 s |tf(/i)) s 

= (c/2) 2 (A*|A*) s + c( Ps \Al) s + Wp) + 2p s \#(p)) s + c(Al\#(p)) s 
= C+ (#(//) + 2p s |tf(//)) s + c\p°\. 

Similarly, arguing as in (|4.9|) . we have 

(0(lA) + 2p a \0(ji)) a = [y + 2 Pl \v) x . 



This completes the proof of the first identity in the lemma. 
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5. KOSTANT HOMOLOGY FORMULAS FOR LlE SUPERALGEBRAS 

5.1. The u_-homology groups of g-modules. Recall the dual pair (q,G) of type 
r G {a, b, c,t>} in Section [221 the Levi subalgebra [ of g, and 7^ , the set of dominant 
weights for [. We denote by L(l, fi) the irreducible highest weight [-module with highest 
weight n G f)*. The following results for integrable modules of Kac-Moody algebras 
apply to our setting. 

Proposition 5.1. {cf. |GL] . [Ju] Prop. 3.1], |Liu[ Prop. 18 and Lemma 20]) Let A G 
?{G). 

(1) // a weight rj G J 3 ^ with (77 + 2p c \r]) c = (A r (A) + 2/3 c |A r (A)) c appears in A fc u_ C§ 
L(g, A r (A)), i/zen r/ = w o A r (A) /or some ttj G ^^(p) 77 appears with multi- 
plicity one. 

(2) The {-module Hfc(u_; L(g, A r (A))) is completely reducible. Moreover, if L(l, 77) 
is a summand of Hfc(u_; L(g, A r (A))), i/ien (77 + 2/) c |r/) c = (A- r (A) + 2p c |A f (A)) c . 

For A G ?(G) and 7tj G we put 

'l( , A a (A)) L(g, A a (A)), if G = 0(2*), 



£(g,Af(A)): 
(5.1) 

£((,woAf(A)) : 



L(g, A r (A)), otherwise, 

' L(l,w o A a (A)) © L({,w o A a (A)), if G = 0(2£), 
L([, ti> o A r (A)), otherwise. 



5.2. The u -homology groups of g-modules. Recall the dual pair (g, G) of type 
r G {a, b,c, t)} given in Section [231 For /1 £ f) we denote by L(l, fi) the irreducible 
highest weight I- module with highest weight \i. 

Lemma 5.2. Let A G J' 9 (G). T/ie {-module L(g, A^(A)) is completely reducible. 

Proof. Suppose that g = g{(p + m|g + n). By |CWi~l Theorem 3.2] (also cf. [S2]). 
S(C p l 9 * © C d *) and S(C m l n ® C d ) are completely reducible over fll(p|g) and g[(m|n), 
respectively. Indeed 5(C m ' n ® C d ) lies in the semisimple tensor category of finite 

dimensional gl(m|n)-modules with composition factors of the form L(g[(m|n), X^), for 
some A G with X m+1 < n ([CK] Theorem 3.1]). Also, S{C p \ q * © C d *) lies in a 
similar semisimple tensor category 0t+*. Thus S^C^ 9 * © C '* © C m l™ © C d ), as a 

gl(m|n) ©gl(p|g)-module, lies in "Qjj~t (8) Crj^ *" , and hence is completely reducible over 

I = g[(m[n) © gl(p|g) © Cif. Therefore L(g, A" (A)) is completely reducible over I. 

If g is of type b, c, 0, then [ = g{(m\n)(BCK. Thus L(g, A^-(A)) is completely reducible 

over I since it is a submodule of S(C m l n © C d ) or S(C m l n © C d ) © A(C^). □ 



Now consider the homology groups of Lie superalgebras Hfc(u_; L(q, A*(A))), which 
are defined analogously (see e.g. [KKj ) . 

Lemma 5.3. Let X G y fl (G). T/ie {-module Hfc(u_; L(g, A'^(A))) is completely reducible. 
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Proof. Suppose that g = gl(p + m\q + n) D [ = g[(m|n) © Qi(p\q) © CAT. It follows from 
[CWT1 Theorem 3.3] that A fc (u-) lies in "0+^ © 0j+*". By LemmaEJ L(g, AJ(A)), 
as an g[(m|n) ffifl[(p|g)-module, lies in "0+t ® 0i+*" , and hence A fc (u_) © L(g, A* (A)) 

is a completely reducible [-module. Since any subquotient of a completely reducible 
module is also completely reducible, the result follows. 

The ortho-symplectic cases are easier and hence omitted. □ 

The proof of Lemma l5.3l implies that if L(J, 7) is an I-submodule of Hfc(u_; L(g, Aj(A))), 
then 7 G We have the following super-analogue of the action of the Casimir op- 

erator on homology groups (see Proposition 15. ip . 

Proposition 5.4. Let 7 G CP|~ + . 7/L(T, 7) is a summand of Hfc(u_; L(g, A^(A))) ; i/ien 
(7 + 2 Ps \ 1 ) s = (AJ(A) + 2 Ps |A r / (A)) s . 

Proof. The proof follows the same type of arguments as for [Liul Proposition 18] and 
thus will be omitted. We only remark that in the process we use the same bilinear form 
(■\-)s and the same p s to define the corresponding Casimir operator for I as in (|4.3p . □ 

5.3. Formulas for the u -homology groups of g-modules. Recalling $(A r (A)) = 
P f (X) for A G ?0(G), we define £(g, A^-(A)) and £(U(woA f (A))) similarly as in fl5jQ). 

Lemma 5.5. Let A G $*{G) and ft G If L(i,ft) appears in the decomposition 

of A fc ii_ © £(g, Aj(A)) u>ii/i multiplicity m-p, then there exists a unique \i G 3[~ wi/i 
= /i ; and L(l, p) appears in the decomposition of A fc u_ ©£(g, A r (A)) wit/i i/ie some 
multiplicity m-jz. 

Proof. Since w J (ch£(g, A* (A))) = ch£(g, Aj(A)), for A G TS(G), and (J (chA fc (u_)) = 
chA fc (u_), we conclude that 

(J (ch[A fe (u_) ®£( fl ,Af(A))]) = ch[A fe (u_) ®£(5,AJ(A))]. 

Since 1? is a bijection on there exists a unique p G such that = /I. There- 
fore L([, /i) is a composition factor of A fc u_ © £(g, A a (A)), if L(T, is a composition 
factor of A fc ii_ © £(g, A ? (A)). Furthermore they have the same multiplicity. □ 

Lemma 5.6. Let A G and 77 G "Pj^ such that L(J,r]) is a summand of A k (u ) © 

£(g, Aj(A)). TTien (77 + 2p s |?7) s = (A^(A) + 2p s \A*J\)) s if and only if there exists 
w G W£(y) with T) = $(w o A ? (A)). 

Proof. Assume that there exists w G WS (y) with woA r (A) G J 3 ^. Set 77 = ^(t/joA^A)). 
Then by Lemma 14.21 and Lemma 14.31 and the VF-invariance of (-|-) c we have 

(77 + 2 Ps \ v ) s = ±(w o A r (A) + 2 Pc \w o A r (A)) c + C 

= ±(A r (A) + 2 Pc |A?(A)) c + C = (A r f (A) + 2p s \A*JX)) s . 
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On the other hand suppose that (77 + 2p s \rj) s = (A^(A) + 2p s |A^(A)) s . By Lemma 1531 
there exists a unique 7? _1 (r/) G "P^ with (r?)) = r\ satisfying the conditions of 

Lemma 15.51 By Lemma 14.21 and Lemma |4.3[ we have 

fa) + 2p c \d~\n)) c = (A r (A) + 2p c |A*(A)) c . 
By Proposition 15.11 (1) there exists w G W5(y) such that 7?~ 1 (r/) = w o A r (A). □ 

Let TV^(p)' denote the subset of W°(p) consisting of those w with #(u;oA r (A)) G !P^ + . 
The proof of the following theorem is inspired by [Ar] . 

Theorem 5.7. Let A G ^{G), k G Z+, and d be as before. As [-modules we have 
Hfc(u_;£(g, A^(A))) = £(U( W oA^(A))). 

7n particular, ch[H fc (u_; £(g, A^(A)))] = w r (ch[H fc (u_; A r (A))] ) . 

Proof. Let /j G be such that L(l, p) is a summand of Hfc(u_; £(g, A y (A))). Then it is 
precisely a summand of A fc u_ <g> £(g, A- r (A)) with (p + 2p c \p) c = (A r (A) + 2p c |A r (A)) c by 
Proposition ^. 11 Furthermore each appears with multiplicity one or two (cf. Remark 5.2 
|CK| ). By Lemma [531 the corresponding $(/i)'s, for p G iP^ + , are precisely the weights 
in "?^ + such that L(l,i}(p)) appear as summands of A fc iL_ ® £(g, A'^(A)) with ($(/j) + 

2p s \ , d(p)) s = (Aj(A) + 2p s |A^(A)) s ; moreover it appears with the same multiplicity. 
Theorems 13. 1\ 13, 4[ and 13.51 together with the Euler-Poincare principle imply that 

oo oo 

X)(-l) fe ch[H fc (u_;£(0,A J / (A)))] =^(-l) k £ 

fe=0 fc=0 iueW°(y)' 

where 

' fc SA + (4, i/)/!^ (77- 1 , x- 1 ), ifG = GL(d), 
^A,»= Kfei), if G = Sp(d), 0(2* + 1), Pin(d) , 

>U^x) + /^Jr7,*), if G = 0(21). 
= ch£(I,i9(«;oA 3: (A))). 
Since all the highest weights are distinct, we conclude from Proposition 15.41 that 
<&[H fc (u_;,CC0,A*(A)))] = Yl ch[£([,^oA^(A)))], 

which is equal to w r (ch[H fc (u_; £(g, A ? (A)))] ) by (12TT2D . □ 
Corollary 5.8. 

(1) T/ie character o/H&(u_; L(g[(p+m|g+n), A"(A))), i.e. the trace of the operator 
I\i,j, s ,t x f" r lj : '' J yf S3 (,t t ' t > is 9iven by 



i«etf»(a)' 
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(2) The character o/H fc (u_; L(spo(2m|2n+l), A^(A))), i.e. the trace of "[[^ rjf* xf j , 
is 

d 

(3) The character of H fc (u_; L(osp(2m|2n), A^(A))), i.e. the trace of 'X[ i ^'nf i x^ j , 
is 

d 



Y\ ( — — — j hs \ w (V, x ) 



(4) The character of H fe (u_; L(spo(2m\2n), A^(A))) ; i.e. the trace of JT- nf % x^ 3 , 



is 




?Zl^HL ) 2 hs^fa X ), if G = 0(21 + 1), 

— 

" " / [hs Xw ( V ,x) + hs~ x Jn,x)], ifG = 0(2£). 

These character formulas of Hfc(u_; A^(A))) fit well with those of g- modules in 
Section [3] by the Euler-Poincare principle. 

Theorem 15.71 and Lemma 15.61 also imply the following super analogue of Proposi- 
tion [5J] (2), which is the converse of Proposition 1ST 



Corollary 5.9. IfL(l,rj) with (r] + 2p s \rj) s = (A^(A) + 2p s |A^(A)) s appears in A fc (u_)® 

£(5, Ay (A)) with multiplicity m^, then L(l,rj) is a summand o/H/^uL; A^(A))) with 
the same multiplicity m^. 

Remark 5.10. Specializing to q = n = in Corollary 15.81 (1), (3) and (4) we obtain 
Kostant type homology formulas of the unitarizable highest weight modules of the Lie 
groups SU(p, m), SO* (2m), and the double cover of Sp(2m, R). For unitarizable highest 
weight modules such homology groups were first computed by Enright |En} Theorem 
2.2], and they involve a complicated subset of the corresponding finite Weyl group 
[DES, Definition 3.6]. Corollary 15.81 provides an alternative description involving an 
infinite Weyl group. Ngau Lam has informed us that both forms of the formula are 
equivalent [HLT]. 

Remark 5.11. Although all the discussions above dealt with u_-homology and u_- 
homology groups, our calculations also give formulas for the corresponding (restricted) 
u + -cohomology and u + -cohomology groups (see |Liul Lemma 9]). 

6. Homology formulas for oscillator modules at negative levels 

In this section, we shall compute the character formulas and the u_-homology groups 
of various modules of b^, Coo and Doo at negative levels. As the approach is parallel to 
that of the earlier sections, the presentation here will be rather sketchy. 
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6.1. The character formulas. We fix a positive integer i > 1. Consider t pairs of 
free bosons 



±,% z -r-\/2 



with i = 1, ...,£. Let denote the corresponding Fock space generated by the 
vaccum vector |0), which is annihilated by 7^'* for r > 0. We also denote by 5 r_ ^ + 2 
the tensor product of and J 2 . 

6.1.1. The case of d^. Suppose that fj = and d is even. By [Wat Theorem 5.2] 
there exists a commuting action of t)oo and Sp(cZ) on $~ 2. Furthermore, under this 
joint action, we have 

(6-1) H= L(d 00 ,A d _(X)) <£> ^sp(d)' 

Aey(s P (d)) 

where A a _(A) := -dAg + ELi A fc e fc . 

Computing the trace of the operator rineN^n" Wi=i Z T on both sides of (|6.1j) . we 
obtain the following identity: 

d 

( 6 - 2 ) nil a _, z) l_ x z -u = E chL( 9oo ,A°_(A))ch^ p(d) . 

By (|2.5|) and similar arguments as in Theorem I3.lti3.5t we obtain the following. 
Theorem 6.1. For A G CP(Sp(d)), we have 

E£L E w gW0(c)(- 1 ) fcs (A„)'(^1^2, • • •) 



chL^A^A)) 



Ui<j0- -XiXj) 



The involution u) on the ring of symmetric functions in x\, X2, ■ ■ ■ maps the left-hand 
side of $2J)$) to that of (JS2]). Thus it follows that ui (chL(coo, A C (A))) = chL(9 00 , A^(A)). 

6.1.2. The case o/Cqq. By [Wat Theorem 5.3 and 6.2] there exists a commuting action 
of Coo and 0(d) on 3" 2 . Furthermore, under this joint action, we have 

(6-3) $~i= L( Coo ,AL(A))®y A (d) , 

Ae0>(O(d)) 

where A C _(A) := -f Aq + ELi h^k- 

If d = 2^, then the calculation of the trace of the operator IlneN x n n Ili=i Z T on 
both sides of (|6,3p gives the following identity 

(6-4) I[n fl _ x z '_ x z -u = £ chL(c_Al(A))chF V). 
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If d = 21 + 1, then the trace of the operator J^neN x n" Y\i=i z T^~^d) gives 
(6-5) " ' 

1 1 

II II 77Z ^TTZ T = E chL( Coo ,AL(A))chy A (2 , +1) . 

i=lnGN^ eX n z i)\L ex nZi A 1 ^ x n) X &{0{21+1)) 

Applying the same arguments as in Theorem 16.11 to (|2.7p and (j2.8|) . we obtain the 
following. 

Theorem 6.2. Let A G ?(O(c0) 6e given. 
(1) If d = 21 + I, then we have 

E£Lo E^eT^Co)^ 1 )^™)'^ 1 '* 2 ' • • •) 



chL( Coo ,A c _(A)) 



(2) If d = 21, then we have 

chL( Coo ,A c _(A)) +chL( Coo ,A c _(A)) 

Efc°=o Ewew^)(- 1 ) k [s(\ w y (xi,X2, ■ ■ ■) + x 2 , ■ ■ ■)] 

UiKji 1 -XiXj) 

Note that u maps ([12]) (resp. {23} ) to (gOED (resp. (^3]) 1. 
6.1.3. TTie case o/bj^. Let d be even. Recall the duality from Theorem lA.31 Computing 

~ d 

the trace of the operator flneN x n n Ylf=i Z T on both sides of (|A.9P gives the following 
identity 



i i 

" 2 ' 



By (|2.1ip and similar arguments as in the proof of Theorem 16.11 we obtain the fol- 
lowing. 

Theorem 6.3. For A G CP(Pin(d)) ; we have 

chL(b DO , A_ (A)) - mi + Xl) -i Ui ^- XlXj) • 

Now uj maps the left hand side of (|2,lip to that of (|6.6p . Thus it follows that 
u (chL(boo,A b (A))) = dhL(aSo,AL (A)). 

6.2. Formulas for the u -homology groups. Recall and from Section 14.31 
and Appendix IA. 21 which we shall now denote by O 5 /"' 1 and 7^'* to keep track that [ is 
a subalgebra of j:^ for p G {b, c, c)}. 

Let fx = cA + Xli>i ft e i ^ / be given. Define a bijective map i? : IP^ C — )• 3^1 2 c 

(6.7) ^):=-2cAl+J2^i- 

i>l 
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In particular, we have i9(A±(A)) = Ai(A), for A G ( ?(Sp(d)), where it is understood 
that A r + (A) = A r (A) for y G {c, 0}. 

For p = 2cA b + J2i>i fHU G IPg, we define : 0>J£ — ► O 5 ^ 

(6.8) ^( M ):=-cAg +^Mk. 

i>l 

We have i?(A b (A)) = AL°(A). 

For A G 7(G) and u> G TV , we define L(g, A r _(A)) and ^([.^(woA^A))) similarly 
as in (15. ip . Then it is easy to see that w(ch£(coo, A^(A))) = chC^, A^(A)), and 

w(ch[A fe u_ ®£(Coo,A^(A))]) = ch[A\u ® £(8oo, A^(A))]. 

Using analogous arguments as in Lemma[53J we can check that for p G y^' c , L(l,p) 
is a component in A fc u_ <g) £(coo, A±(A)) if and only if L(l, $(p)) is a component in 
A fe u_ ® ^(Doo,A^(A)) with the same multiplicity, while w(chL([, /i)) = chL([, 
We have a similar correspondence between [-modules inside A fc u_ ® L(boo, A b (A)) and 
A fc u_® A?. 8 (A)). 

Lemma 6.4. For G J 3 ^' 7 , we have 

(p + 2p c |/i) c = -(<%) + 2p c |f?(M)) c> J G {b, c}. 
/n particular, we have 

(1) (A|(A) + 2p c |A^(A)) c = (/x + 2p c |/x) c j/ and on/y if {A\(X) + 2p c |A^(A)) c = 
{®(p) + 2p c \V{p)) c . 

(2) (A b (A) + 2p c |A b (A)) c = (p + 2p c \p) c if and only if (A 6 _°(A) + 2p c |A b !(A)) c = 
(0Gu)+2p c |0(/x)) c . 

Proof. Let /i = cAq + X^i>i A 4 * 6 *- Consider first the case of y = c. We have 
(p + 2p c |/x) c = (y] pi€j + 2p c | ^J/Xjei) c + 2c(Aq| ^/xjej) c 

i>l i>l i>l 

= ^piipi - 2i) - 2c\p°\ = (p° + pi\p°)i - 2c\p°\, 
i>l 

where p° = (p±,p2, • • •)• On the other hand, we have 

i>l %>1 i>l 

i>l 

= -(// + pi|/i°)i + 2c|/z°| = -0 + 2p c |/i) c . 

The identity (|4.7p was used in the second last identity above. 
Now let y = b. One shows that 

(p + 2p c | M ) c + (0(/i) + 2p c |tf(/i)) c = £ - 2» + 1) + J2 M - 2* + 1)- 

i>l i>l 

Now (|4.7|) says that the right-hand side is zero. □ 
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Theorem 6.5. Let k £ Z+, and •& as in (|6.7p or (|6.8p , VFe /taue the following isomor- 
phisms of [-modules: 

HfcCu-jL^oo.ALCA))^ If^^oA^A))), AeO>(Sp(d)). 

HfcCu-^Ccoo.A^CA))^ ^([.^HoA^A))), AG?(0(d)). 

H^L^A^A))^ I(I,^oA b (A))), A G 3>(Pin(d)). 

In particular, we have ch[Hfc(u_; L(doo, A^(A))] = w(ch[Hjt(u_; -L(Coo, Aj_(A))] ) and 
ch[H fc (u_;L(l£„A?!(A))] =^(ch[H fc (u_;L(b 00 ,A fa (A))]). 

Proof. The result follows from the same type of argument as the one used in the proof 
of Theorem 15,71 now using Lemma 16.41 We leave the details to the reader. □ 

Appendix A. Two new reductive dual pairs 

A.l. The (spo(2m|2n + l),Pin(d))-duality. Let d = 2£ be even. 

There exists a commuting action of spo(2m|2ra+l) and Pin(d) on 5* (^ 2m \ 2n + l g) ^2 j 
as follows. We have 

5(C 2m|2n+1 ® Ci) = 5(C m|n ® C d ) ® 5(C 011 ® Cl). 

On S^C" 1 '™ (g> C d ) we have an action of the Howe dual pair (spo(2m|2n), 0(d)) by 

Proposition On the other hand on 5(C Q|1 <g> ci) = A(cf) the Lie algebra so(d) 
acts by two irreducible spin representations, giving rise to an irreducible representation 
of Pin(ti). Since representations of 0(d) pull back to representations of Pin(d) we obtain 
a commuting action of spo(2m|2ra) and Pin(d) on 5(C 2m l 2n + 1 <gj C2). This action of 
Pin(d) does not factor through 0(d). Furthermore the commuting action of spo(2m|2n) 
extends to a commuting action of spo(2m|2n + 1). It follows from Section [A. 1.11 and 
arguments similar to |CZ2|, Proposition 4.1] that S(C 2m \ 2n+1 ® C2) is a unitarizable, 
and hence a completely reducible, spo(2m|2n + l)-module. 

A. 1.1. Formulas for 50(d)- and 5po(2m\2n + 1)- action on s(C 2m ^ 2n+1 ®Ca). We intro- 
duce even indeterminates x := {xf, xf} and odd indeterminates £ := > Col) where 

1 < i < m, 1 < j < n, 1 < k < I We identify S(C 2m \ 2n+l <g> ci) with the polynomial 
superalgebra C[x, £]. Then the actions of spo(2m[2n + 1) and so(d) may be realized as 
differential operators as follows. 



Formulas for the so(d)-action. 
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r\ r\ >b Q r\ Tib 

< A - 2 > 4^+E(fe-£j*)+E( a 



d 



.• 



' 9a? c 



r), i<i,j <t;i^j, 



n d 



5 m 

^)+E 



s=l 



dxi 



4± 
s dx\ 



• Formulas for the spo(2m\2n + l)-action. 

(A.3) 

^ I diTTT + xf-JL: I + ^m, Ki,j <m, 



4 i :=V f4^+^4) . Ax . x := ^ f StJ^i- + TTT-!rT I , Ki,j<m, 
(A.4) 

^:=E(ef3 + ^). A ^ : =E( Jra + di^)' i<*.i<T»;i^i- 

fc=l fe=l \ ? i C*?! 



E ( ^ + ti ^Tfc ) ' A ?o& : = E 

fc=l V / fe=l 



9 9 . ^ d 



^:=E(^+^), ^:=E(^ + ^), l<i<m, 

a x . £ . : =y( A + A /:E := y f^.+^f) ,i<i< m ,i<j<n 

(A.5) 

§Nr*# s(^ +f ^)' ^«^^^- 

A. 1.2. TTie module decomposition. It is evident from (|A.1|) that the action of Pin(d) 
on C[x, £] does not factor through 0(d). Since Endc(C[x, £]) Pm ( rf ) is generated by 
spo(2m|2n + 1) in Section lA.1.11 it follows from the double commutant theorem that 
with respect to the spo(2m|2n + 1) x Pin((i)-action we have 

(A.6) C[x,C] = 0L(spo(2m|2n + l),A^(A))^y p A in(d) , 

AeA 



31 



CHENG, KWON, AND WANG 



where A C 3 3 (Pin(d)), and : A — > fj* is an injection. It remains to determine the set 
A and the map Ay. 

Theorem A.l. As an spo(2m|2n + 1) x Pin(d) -module we have 

S{C 2m \ 2n+1 ®d) - 0L( S po(2m|2n + 1), Aj(A)) V P \ n(d) , 

A 

where the summation is over all A € J > (Pin(<i)) with A m +i < n, and Ay-(A) = A^ + ^l m | n . 

Proof. Let u + be the algebra generated by the A-operators, i.e. A x . x , A^., A^., 
A^ QXi , and A Xi ^. . Then u_i_ is invariant under the adjoint action of Ql(m\n). 

An element / £ C[x, £] is called harmonic, if / is annihilated by u + . The space 
of harmonics will be denoted by "K and it evidently admits an action of Ql(m\n) x 
Pin(ci). Furthermore, since S(C 2m \ 2n+1 ®Cz) is a completely reducible g[(m|n)-module, 
L(spo(2m|2n + l),/i) u+ is also completely reducible over gl(m\n), for any irreducible 
spo(2m|2ra + l)-module L(spo(2m|2n + l),/i) that appears in S(C 2m \ 2n+l d). By 
irreducibility of L(spo(2m|2n + l),/i) we must have 

L(spo(2m|2n + 1), /x) n+ L(fll(m|ra), /i). 

So, by (|A.6P (Pin(d),g[(m|n)) forms a Howe dual pair on IK. Thus, proving the theorem 
is equivalent to establishing the following decomposition of "K as a g[(m|n) x Pin(<i)- 
module: 

(A.7) <K * 0L( fl r(m|n), Aj(A)) V P x m(d) , 

A 

where the summation is over all A € !P(Pm(d)), i.e. £(X) < with A m +i < n, and 
A)(A) = Al + *l m |„. 

We first consider the limit case n = oo with the space of harmonics denoted by !K°° . 
Here the only restriction on A is £(X) < £, and we observe that the vector given in 
[CWH Theorems 4.1 and 4.2] associated to such a partition A is indeed annihilated by 
u + and hence is a joint Ql(m\n) x Pin(<i)-highest weight vector of weight (A, A^ +£l m i n ). 
Hence all the summands on the right hand side of (|A.7p occur in the space of harmonics, 
and in particular, all irreducible representations of Pin(d) occur. Therefore, we have 
established (|A.7j) in the case n = oo. 

Now consider the finite n case. We may regard S(C 2m l 1+2n <g>ci) C S(C 2m l 1+2oo «>ci) 
with compatible actions spo(2m|l+2n) C spo(2m|l+2oo). From the formulas of the A- 
operators we see that "K C "K 00 . Thus "K is obtained from "K 00 by setting the variables 
= £j = 0, for j > n. However, it is clear, from the explicit formulas of the joint 

highest vectors in that, when setting the variables = £j = f° r 3 > n > precisely 
those vectors corresponding to A with A m +i < n will survive. □ 



Remark A. 2. Theorem lA.il is a finite-dimensional analogue of |CW2l Theorem 8.2]. 
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A. 2. The Lie superalgebra b^ and the (b^, Pin(d))-duality. Consider the su- 
perspace with basis {vi,Vk\k G Z} with degVk = 0, for k G Z, and degvi = 1. 

2_ 2 

The Lie superalgebra bgo is the subalgebra of the general linear superalgebra preserv- 
ing the even super-skewsymmetric bilinear form determined by (vi\vj) := (— iySi x—j, 
i,j G Z, and (vilvi) := 1. Now consider the central extension of general linear su- 

2 2 

peralgebra corresponding to the 2-cocycle j(A,B) := Str f (X^<i -^jl [A,B]\ where 
Str(cjj) := — cii + Ylij&z c jj- The Lie superalgebra is the central extension of bgo 
by a one-dimensional center CRT, obtained via restriction of the cocycle 7. 

By construction one sees that Cqo C b^, and also Cqo and b^ share the same Cartan 
subalgebra f). This allows us to identify their Cartan and dual Cartan subalgebras. For 
convenience of the reader we list below the simple roots and coroots for b^. 

n v =K =E 1 + K, aX = Ei- E i+1 (i G N)} 

n ={a = -ex, Oi = ei- e i+1 (i G N)} 
A + ={±€i - ej, -a, -2ei (i G N,i < j)}. 
The associated Dynkin diagram is as follows: 



•^o— o c^o— o- • • • 

OtQ OL\ Ot2 &n — l Oift CK n -|_l 

Let p c ,A b ° G b* be determined by {p C) Ej) = -j + §, {p c ,K) = (A b °,Ej) = 0, 
and (Aq° ,K) = 1, j G N. For c G C let consist of elements in b* of the form 

cA 0° + Si>i Mi 6 *' where (jUi,/i 2 , . . .) G IP+. 

In Section [6] the following bilinear form (-|-) c on b^ is used. We first choose a bilinear 
form (-|-) c on b* satisfying 

{X\e i ) c = (X,E i -K) 1 i G N, 

= « 1^ = 0. 

One checks that (ej|ej) c = (Ag°|ej) c = —1 for i, j G N, and 2(p c \ai) c = (oii\oti) c for 
i G Z + . Let { }jgz+ be the sequence defined by 

6 o _ J -1, if i = 0, 
Sj ~\l, if i > 1. 

It follows that by defining sfs^ (a^|aj) c := (aj|aj) c , we obtain a symmetric bilinear 
form on f). This form can be extended to a non-degenerate invariant super-symmetric 
bilinear form on b^ such that 

(A.8) (ei\f j )c = S ij /sT, 

where e% and fj G Z+) denote the Chevalley generators with [ej, /i] = a^. Now we 
can define the Casimir operator O as in \CK\ Section 4.1]. 
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Let d = 2£ be even. By \Wa\ Theorem 5.3] there exists a (Cooj 0(d))-duality on 

Now A(C2) is an irreducible Pin(d)-module and hence we have a commuting action of 

Coo and Pin(d) on 3"2 ® A(Ca). One can show that the action of Coo on 3"2 <S> A(Ca ) 
extends to an action of b^, that commutes with the action of Pin(d), and furthermore 

b^ generates End(3~! g> A(ci)) Pin ^. 

In the sequel we will need to have this commuting action in a more explicit form. For 
this let us introduce odd indeterminates £i := {£i\k = !,...,£}, and identify A(Cz) 



2 



with the Clifford superalgebra generated by £i. The action of the Lie algebra so(d) 

d 2 

on A(Ca) in terms of differential operators in £i is explicitly given by the summands 

2 

involving £q in (|A. 1[) and (|A.2p . This action combined with the action of so(d) on 5 ^ 
as in \Wa\ (5.52)], gives the action of so(d) on 5 2 ® A(Ca). The commuting action of 
b^-, is as follows. First the subalgebra Cqo acts on 3" 2 (g> A(C2 ) only on the first factor 
as in [Wat Section 5.2]. To complete the description we only need to give formulas for 
the action of the odd root vectors h , and A* 1 j, corresponding to the roots q and — e,, 

i G N, respectively. They are as follows: 



fe=i \ 2 / " fc=1 

We have the following. 

Theorem A. 3. As a b^ x Pin(cZ) -module we have 

(A.9) H®A(d)- L^A^A))®^ 

Ae!P(Pin(d)) 



w/iere A^°(A) := -§ Ag° + £* =1 W 

Proof. One checks that the Cqo x so((i)-joint highest weight vectors inside §"2 given 
in [Wat (5.54)] are also b^-highest weight vectors. For this it is enough to check that 

they are annihilated by the root vector A^ 1 i (corresponding to the simple odd root 

s 

— ei). This, however, is easy. The duality then follows from the fact that this set of 
vectors exhaust all irreducible finite dimensional Pin(d)-highest weights that do not 
factor through 0(d). □ 
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